The invention relates to a multi-aperture device and to a method for detecting an object region.
Conventional cameras are based on the single-aperture principle. A continuous image of the object is taken here, wherein neighboring regions in the object are also neighboring in the image. In order to record depth information, in most cases two cameras exhibiting a lateral distance to each other are used (stereoscopy). The cameras view the object from different angles (parallax), in correspondence with the distance of the cameras (base length) and the object distance. Consequently, the images of the two cameras are shifted laterally to a different extent to each other (disparity), depending on the object distance. The object distance may be concluded when knowing the base distance of the cameras and the disparity. Increasing the base length advantageously results in an improved depth resolution or a way of differentiating depths in greater object distances. The desired miniaturization of the entire camera setup, however, is counteractive. Furthermore, in practical realizations, the precision of the depth information is limited by the precision of positioning the two cameras relative to each other (precision of base length).
Alternatively, there are array systems pursuant to the super-resolution principle, as are, for example, described under the term Pelican Imaging camera-array (PiCam) in [1]. Such cameras may consist of a plurality of individual imaging channels (2×2, 3×3, 4×4, 5×5, etc.) which each generate an image of the entire object. Generally, such multi-channel setups may be set up in arrangements of N×N or N×M, N and M being greater than or equal to 2. Since the individual cameras of the array exhibit a lateral distance to one another, the same parallax effect results and information on object distances may be gained by means of image processing. Since the array camera as a whole is to be small, for example due to integration in mobile devices, like mobile phones, smartphones, notebooks, etc., the distance between the individual channels of the array camera is small and, consequently, the depth resolution is strongly limited. A goal of the multi-channel setup is reducing the structural height of the camera. A smaller structural height may, for example, be achieved by reducing the focal length f of the imaging channels.
A light-sensitive pixel of a width dp of a camera receives light from the angular region a=arctan(dp/f), wherein the angular region a is also referred to as angle of acceptance. When decreasing the focal length f, the result is a greater angle of acceptance a. Increasing the angle of acceptance a is equivalent to decreasing the resolution, since only fewer object regions may be differentiated among one another. In order not to suffer from a loss in resolution when decreasing the focal length, the principles of super-resolution are applied. A pre-requisite here is aliasing to be present, that is the optics generates point images which are smaller than the pixel pitch, that is the distance between two pixels. The fields of vision of the individual cameras here are shifted by fractions of a pixel pitch. When the width of the point image is smaller than the pixel pitch by the factor N, the fields of vision of the cameras are each shifted by an N-th of the angle associated to a pixel. This means that the optical axes of the individual cameras are each shifted by an N-th of the pixel pitch. The shift here may be performed in the X- and Y-directions, that is there may be N2 cameras with respective sub-pixel shifts. A high-resolution overall image may then be calculated from the sub-scanned sub-images with a sub-pixel offset using image processing in software.
A further alternative are cluster eye cameras (see DE102009049387), as are exemplarily discussed in FIGS. 9 and 10. Cluster eye cameras may, similarly to the array cameras, consist of a plurality of individual channels. However, the channels of the array do not transmit the entire object field, but the individual channels each only view sub-regions thereof. The sub-images of the respective object sub-regions in turn are united to form an overall image by means of image post-processing. The lateral arrangement of the imaging channels with a respective associated visual field sub-region on an image converter is as desired. In contrast to classical single-aperture cameras and even array cameras which really are to be understood to be array arrangements of conventional single-aperture cameras, in these setups the visual field and the lateral position of the image on the electronic image converter are decoupled from each other. In order to reduce the structural height, small focal lengths are used again. The super-resolution method including sub-images shifted by sub-pixels is used here in order not to suffer from a loss in angular resolution.
In a present solution, the imaging channels are arranged such that laterally neighboring channels are also neighboring in the angle of the visual field. The result of this is that, due to the small base distance of the channels, only a small parallax occurs and thus only small shifts of the sub-images which also result in a limited depth resolution may be observed. In order to be able to evaluate the shift of the sub-images, these are to be overlapping partly, that is contain equal image contents in individual regions. The distance between the object and the camera may be concluded from comparing the lateral position of the sub-regions of identical contents in the corresponding imaging channels, when knowing the base length. Since this may be performed for each image point, a depth chart of the object space may be established.
Consequently, the object underlying the present invention is providing a device and a method allowing an object region to be detected with improved depth information.